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Abstract — In this work, we study the problem of reconstructing 
a sparse signal from a limited number of linear 'incoherent' 
noisy measurements, when a part of its support is known. 
The known part of the support may be available from prior 
knowledge or from the previous time instant (in applications 
requiring recursive reconstruction of a time sequence of sparse 
signals, e.g. dynamic MRI). We study a modification of Basis 
Pursuit Denoising (BPDN) and bound its reconstruction error. 
A key feature of our work is that the bounds that we obtain 
are computable. Hence, we are able to use Monte Carlo to 
study their average behavior as the size of the unknown support 
increases. We also demonstrate that when the unknown support 
size is small, modifled-BPDN bounds are much tighter than those 
for BPDN, and hold under much weaker sufficient conditions 
(require fewer measurements). 

Index Terms — Compressive sensing, Sparse reconstruction 



I. Introduction 

In this work, we study the problem of reconstructing a 
sparse signal from a limited number of linear 'incoherent' 
noisy measurements, when a part of its support is known. 
In practical applications, this may be obtained from prior 
knowledge, e.g. it can be the lowest subband of wavelet 
coefficients for medical images which are sparse in the wavelet 
basis. Alternatively when reconstructing time sequences of 
sparse signals, e.g. in a real-time dynamic MRI application, it 
could be the support estimate from the previous time instant. 

In [3], we introduced modified-CS for the noiseless mea- 
surements' case. Sufficient conditions for exact reconstruction 
were derived and it was argued that these are much weaker 
than those needed for CS. Modified-CS-residual, which com- 
bines the modified-CS idea with CS on LS residual (LS- 
CS) [5], was introduced for noisy measurements in [4] for 
a real-time dynamic MRI reconstruction application. In this 
paper, we bound the recosntruction error of a simpler special 
case of modified-CS-residual, which we call modified-BPDN. 
We use a strategy similar to the results of [2] to bound the 
reconstruction error and hence, just like in [2], the bounds we 
obtain are computable. We are thus able to use Monte Carlo to 
study the average behavior of the reconstruction error bound 
as the size of the unknown support, A, increases or as the 
size of the support itself, N, increases. We also demonstrate 
that modified-BPDN bounds are much smaller than those for 
BPDN (which corresponds to | A| = \N\) and hold under much 
weaker sufficient conditions (require fewer measurements). 



In parallel and independent work recently posted on Arxiv, 
[7] also proposed an approach related to modified-BPDN 
and bounded its error. Their bounds are based on Candes' 
results and hence are not computable. Other related work 
includes [8] (which focusses on the time series case and mostly 
studies the time-invariant support case) and [9] (studies the 
noiseless measurements' case and assumes probabilistic prior 
knowledge). 



A. Problem definition 

We obtain an rt-length measurement vector y by 



y = Ax + w 



(1) 



Our problem is to reconstruct the m-length sparse signal x 
from the measurement y with m > n. The measurement is 
obtained from an n x m measurement matrix A and corrupted 
by a n-length vector noise w. The support of x denoted as N 
consists of three parts: N = T U A \ A e where A and T are 
disjoint and A e C T. T is the known part of support while 
A e is the error in the known part of support and A is the 
unknown part. We also define N e = T (J A = N (J A e . 

Notation: We use ' for conjugate transpose. For any set T 
and vector b, we have (6)t to denote a sub-vector containing 
the elements of b with indices in T. \\b\\k means the Ik norm 
of the vector b. T c denotes the complement of set T and is 
the empty set. For the matrix A, At denotes the sub-matrix by 
extracting columns of A with indices in T. The matrix norm 
1 1 -A ||p, is defined as 

P || p A 

x^O \\x\\p 

We also define 5s to be the S'-restricted isometry constant 
and 8s,s' to De the S, S' restricted orthogonality constant as 
in [6]. ' 



II. Bounding modified-BPDN 

In this section, we introduce modified-BPDN and derive the 
bound for its reconstruction error. 



2 



A. Modified-BPDN 

In [3], equation (5) gives the modified-CS algorithm under 
noiseless measurements. We relax the equality constraint of 
this equation to propose modified-BPDN algorithm using a 
modification of the BPDN idea[l]. We solve 



min \\\y-Ab\\l +7||bHli 

L 



(2) 



Then the solution to this convex optimization problem x will 
be the reconstructed signal of the problem. In the following 
two subsections, we bound the reconstruction error. 

B. Bound of reconstruction error 

We now bound the reconstruction error. We use a strategy 
similar to [2]. We define the function 



L(b) = \h 



A6||^+7||&tc||i 



(3) 



Look at the solution of the problem (|2} over all vectors 
supported on N e . If A^ c has full column rank, the function 
L(b) is strictly convex when minimizing it over all b supported 
on N e and then it will have a unique minimizer. We denote 
the unique minimizer of function Lib) over all b supported on 
N e as 



b=[b' Ne 0' ] 



(4) 



Also, we denote the genie-aided least square estimate sup- 
ported on N e as 



Since 



-X\\2 



0' N c] where c Ne := (A' N AjvJ -1 ^ 



N e y 



(5) 



< 



IMI 



is quite small if noise is small and 



S\n s \ is small, we just give the error bound for b with respect 
to c in the following lemma and will prove that it is also the 
global unique minimizer under some sufficient condition. 

Lemma 1: Suppose that A^ c has full column rank, and let 
b minimize the function L(b) over all vectors supported on 
N e . We have the following conclusions: 

1) A necessary and sufficient condition for b to be the 
unique minimizer is that 

- 1 (A' T A T )- 1 A' T A A (A' A MA A )- 1 g A 



'A 



7 (^ma a )- 1 5a ) 



and 



Next, we obtain sufficient condition under which b is also 
the unique global minimizer of L(b). 

Lemma 2: If the following condition is satisfied, then the 
problem (f2]) has a unique minimizer which is equal to b defined 
in ©. 

HA'Cy-A^cwJIjoo < 7[l-max ||(^ A M^ A ) _1 A A M^||i] 

The proof of Lemma 2 is in the appendix. 

Combining Lemma 1 and 2 and bounding \\c — x||,we get 
the following Theorem: 

Theorem 1: If Ajy e has full column rank and the following 
condition is satisfied 

\\A'{y-A Nc c N Moo < 7[l-max \\(A' A M Aa)' 1 A' A M AJ^ 

(7) 

then, 

1) Problem (O has a unique minimizer b and it is supported 
on N e . 

2) The unique minimizer b satisfies 

\\b-x\\ 00 < 1 m^(\\(A' T A T )- 1 A , T A A (A' A MA A )- 1 \\ 0C 

and 

||&-.x||2<||(^A jv J- 1 ^J| 2 ||H| 2 +7^- 

-\\(A' A MA A )-m 
(9) 



where M = I - A T (A' T A T ) 
^(II^T c ||i)lb = b- <9(II^t c ||i) is the subgradient set of 
||&t=||i- Thus, g T = and ||g A ||oo = 1. 

2) Error bound in norm 

||6— c||oo <7max(||(^A T )- 1 A! r AA(AA/AA)" 1 !|oo 

JK^AMa)- 1 !^) (6) 

3) Error bound in l 2 norm 

||6-c|| a < ■ 



^WiA^AT^A^A^A^MA-^W 



(8) 



WiA^AT^A^A^A^MA^) 




\w\\ 2 
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|A| 



l-5r. 



(10) 



|JV e | 



Now consider BPDN. From theorem 8 of [2](the same thing 
also follows by setting T = in our result), if An has full 
rank and if 

WA'iy-ANiA'nAtfY^A'ny)^ < 7[l-max||(A' w A JV )- 1 A' Ar ^|| 1 ] 
then Ibpdn 

\\b B PDN- x|]oo < 7ll(^W^A) _1 ||oo + ||(^^A) _1 A' Ar | 



Allool 



Similarly, we can have the l 2 norm bound of BPDN is 

1 IMIa 



\\bBPDN - X\\ 2 < 7V l-^l 



1 



<\N\ \J^- °|A| 

Compare ( TTOb and (fT3l for the case when |A| = |A e | = 
^(follows from [4]), the second terms are mostly equal. 
Consider an example assuming that <Wi = 0.5, <5i A i = 0.1, 
^|T|.|A| = 0-2 and |A| = t^l^l which is practical in 
real data. Then the bound for BPDN is 2j B pdn VWl + 
0.7||ui||2 and the bound for modified-BPDN approximates 
to 1.37 mo rfspDA-|A| + 0.7|H| 2 . Using a similar argument, 
ImodBPDN which is the smallest 7 satisfying ©, will be 
smaller than jbpdn which is the smallest 7 satisfying 
dTTb . Since |A| = ^y|iV| and jbpdn will be larger than 
ImodBPDN, the bound for modified-BPDN will be much 
smaller than that of BPDN. This is one example, but we do a 
detailed simulation comparison in the next section using the 
computable version of the bounds given in (O and (|9]). 



(12) 



(13) 




|T|,|A| 



\(A' A MA A 
1 



(i-<W 2 

The proof is given in the Appendix. 
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|A| 



1 — <5i- 
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III. Simulation Results 

In this section, we compare both the computable and I2 
norm bounds for modified-BPDN with those of BPDN using 
Monte Carlo simulation. Note that, BPDN is a special case of 
modified-BPDN when A = N and A e = 0. Therefore, we do 
the following simulation to check the change of error bound 
when |A| increases and compare the bounds of modified- 
BPDN with those of BPDN. 

We do the simulation as follows: 

1) Fix m = 1024 and size of support \N\. 

2) Select n, |A| and |A e |. 

3) Generate the n x m random-Gaussian matrix, A (gen- 
erate an n x m matrix with i.i.d. zero mean Gaussian 
entries and normalize each column to unit £2 norm). 

4) Repeat the following tot = 50 times 

a) Generate the support, N, of size \N\, uniformly at 
random from [1 : m]. 

b) Generate the nonzero elements of the sparse signal 
x on the support N with i.i.d Gaussian distributed 
entries with zero mean and variance 100. Then 
generate a random i.i.d Gaussian noise w with zero 
mean and variance <x^. Compute y := Ax + w. 

c) Generate the unknown part of support, A, of size 
|A| uniformly at random from the elements of N. 

d) Generate the error in known part of support, A e , 
of size |A e |, uniformly at random from [1 : m]\N 

e) Use T = N U A e \ A to compute 7* by 

* = \\A'{y- An^cnJWoq 

1 - max^AT. UA^MAa^A^MAJ-l 

and do reconstruction with 7 = 7* using modified- 
BPDN to obtain x mod BPDN- 

f) Compute the reconstruction error \\x mo dBPDN — 

C\\oo 

g) Compute the norm bound from (O and the I2 
norm bound from (O. 

5) Compute the average bounds and average error for the 
given n, |A|, |A e |. 

6) Repeat for various values of n,|A| and |A e |. 

Fig[T] shows the average bound(RHS of (O) for different |A| 
when \N\ = 100 ~ 10%m which is practical for real data as in 
[3], [4]. The noise variance is crL = 0.001. We show plots for 
different choice of n. The case U = 1 in Fig. Q] corresponds 
to BPDN. From the figures, we can observe that when |A| 
increases, the bounds are increasing. One thing needed to be 
mentioned is that for BPDN(A = N, A e = 0) in this case, 
the RHS of (|7]i is negative and the bound can only hold when 
number of measurements n > 0.95m. Therefore, BPDN is 
difficult to meet the unique minimizer condition when \N\ 
increases to 0.1m. However, when |A| is small, modified- 
BPDN can easily satisfy the condition, even with very few 
measurements(n = 0.2m when |A| = 0.05|iV|). Hence, the 
sufficient conditions for modified-BPDN require much fewer 
measurements than those for BPDN when |A| is small. Figj2] 
gives another group of results showing average bound(RHS of 
©) for different |A| when |AT| = 15 w 1.5%m. The noise 
variance is erj, = 0.0003 and A e = 0. We can also obtain 



[N|=100,|AJ=0 




°0 02 0.04 0.06 0.08 0.1 

|A|/|N| 

(a) \N\ = 100, A e = 




°0 02 0.04 0.06 08 0.1 

|A|/|N| 



(b) \N\ = 100,|A e | = i|iV| 

Fig. 1. The average boundl(9j on \\b — x\\ 2 is plotted. Signal 
length m = 1024 and support size \N\ = 100. For fixed n and 
|A e |, the bound increases when |A| increases. When number of 
measurements n increases, the bound decreases. When n = 0.2m 
and |A| > 0.05|iV|, the RHS of @ is negative and thus the bound 
does not hold. We do not plot the case of BPDN(A = N, A e = 0) 
since it requires n > 0.95m measurements to make RHS of Q 
positive. 

the same conclusions as FigQ] Note that we do not plot the 
average error and bound for |A| > ||JV| when n = 0.2m 
since the RHS of (O is negative and thus the bound does 
not hold. Hence, the more we know the support, the fewer 
measurements modified-BPDN requires. 

In this case, we also compute the average error and the 
bound © on \\b — c\\oo. Since \N e \ = 15 is small and noise is 
small 1 1 c— x | loo will be small and equal for any choice of |A|. 
Thus we just compare \\b — c||oo with its upper bound given 
in ((6]). For the error and bound on ||fo — c|j when we fix 
n = 0.3m and A e = 0, the error and the bound are both for 
|A| = which verifies that the unique minimizer is equal to 
the genie-aided least square estimation on support N in this 
case. For |A| = ||iV|, the error is 0.08 and the bound is 0.09. 
For |A| = §|JV|, the error is 0.21 and the bound is 0.27. When 
|A| = \N\ which corresponds to BPDN in this case, the error 
increases to 3.3 and the bound increases to 9. Therefore, we 
can observe that when |A| increases, both the error and the 
bound are increasing. Also, we can see the gap between error 
and bound(gap=bound-error) increases with |A|. 

From the simulation results, we conclude as follows: 

1) The error and bound increase as |A| increases. 

2) The error and bound increase as \N\ increases. 

3) The gap between the error and bound increases as |A| 
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|N|=15,|A |=0 



-•-average bound n=0.2m 
-▼-average bound n=0 3m 
♦average bound n=0 5m 




Then, we have 



Fig. 2. The average boundl[9} on ||6 — x\\% is plotted. Signal length 
m = 1024, support size \N\ = 15 and |A e | = 0. For fixed n, the 
bound on 1 1£> — ar| [2 increases when |A| increases. When number of 
measurements n increases, the bound decreases. When n = 0.2m 
and |A| > ^ \N\, the RHS of is negative and thus the bound does 
not hold. 



increases. 

4) The error and bound decrease as n increases. 

5) For real data, \N\ « 0.1m. In this case, BPDN needs 
n > 0.95m to apply the bound while modified-BPDN 
can much easily to apply its bound under very small n. 

6) When n is large enough, e.g. n = 0.5m for \N\ = 15 = 
15%m, the bounds are almost equal for all values of |A| 
(the black plot of Fig. 2) including | A| = \N\ (BPDN). 

IV. Conclusions 

We proposed a modification of the BPDN idea, called 
modified-BPDN, for sparse reconstruction from noisy mea- 
surements when a part of the support is known, and bounded 
its reconstruction error. A key feature of our work is that the 
bounds that we obtain are computable. Hence we are able 
to use Monte Carlo to show that the average value of the 
bound increases as the unknown support size or the size of 
the error in the known support increases. We are also able to 
compare with the BPDN bound and show that (a) for practical 
support sizes (equal to 10% of signal size it holds under very 
strong assumptions (require more than 95% random Gaussian 
measurements for the bound to hold) and (b) for smaller 
support sizes (e.g. 1.5% of signal size), the BPDN bound is 
much larger than the modified-BPDN bound. 

V. APPENDIX 

A. Proof of Lemma 1 

Suppose supp(b) C N e . We know the vectors y — Ac = 
y — ^4jv c cat c and Ac — Ab = An c (&jv e — Cjv e ) are orthogonal 
because A' N (y — An c cnJ = using Thus we minimize 
function L(b) over all vectors supported on set N e by mini- 
mizing: 



F(b) = ^\\A N .c N .-A N .b N .\\i + 'Y\\bT4i 



(14) 



Since this function is strictly convex, then £ dF(b). Hence, 

A' N A Ne b Ne - A' N A Ne c Nc + igNa = (15) 



Since 



cjv c 



Am A 



bN c = j(A' N A Nc ) l g Ne 



(16) 



A' T A T A' t Aa 
A' a A t A' a A a 



By using the block matrix inversion and = 0, we get 

cn c - b Ne = 

Thus, we can obtain the norm bound of error as below: 



^{A^Ar^A^A^A^MA^gA 
1 {A' A MA A )- 1 g A ) 



||&jv c - Civjoo = j\\(A' Ne A Ne ) 1 gN c \\oo 
< 7 max(||(^A T )- 1 ^A A (^A/A A )- 1 . 9A || oc 

IK^M^-Vlloo) 

<7max(||(^A T )- 1 ^A A (A A A/A A )- 1 ||, 30 , 
UA^MAa)- 1 ^) 

This follows using ||.g A ||oo = 1- Also, using ||<7a||2 < 
we get the I2 norm bound of b — c. 



Using \\(A' T A T )- l \\ 2 < 
|^t^a||2 < 0|r|,|A|. we g et CO] 
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\A' A A 



A ^A||2 



> 1-5, 




B. Proof of Lemma 2 

Suppose that An c has full column rank, and let b minimize 
the function L(b) over all b supported on N e = TU A. We need 
to prove under this condition, b is the unique global minimizer 
of L(b). 

The idea is to prove under the given condition, any small 
perturbation h on b will increase function L(b),i.e. L(b + h) — 
L(b) > 0,V||/i||oo < 6 for 6 small enough. Since L(b) is a 
convex function, b should be the unique global minimizer. 

Similar to [2], we first split the perturbation into two parts 
h = u + v where supp(u) = N e and supp(v) = N^. Clearly 
< 5. Then we have 



Mice < 



1 



L(b+h) = ^\\y-A(b+u)-Av\\ 2 2 +j\\(b+u)Tc+v T o\\i (17) 

Then expand the first term, we can obtain 

\\y - A(b + u) - Av\\l = \\y-A(b + u)\\l+ \\M$ 

-2Re(y - Ab, Av) + 2Re(Au, Av) (18) 

The second term of dPTl ) becomes 



\\{b + u) 7 
Then we have 

L(b + h) - 



||(6 + u)t<=||i + ||wT=|h 



(19) 



L(b) = L{b- 



L(b) 



\Av\\\ 



-Re(y - Ab, Av) + Re(Au, Av) + ~/\\v T c 



(20) 



Since b minimizes L(b) over all vectors supported on N e , 
L(b + u) - L(b) > 0. Then since L(b + u) - L(b) > 
and ||Aw||2 > 0, we need to prove that the rest are 
non-negative:7||^c|| x — Re(y — Ab, Av) + Re(Au, Av) > 
0. Instead, we can prove this by proving a stronger one 
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j\\v T 4x-\(y- Ab,Av)\ - \(Au,_Av}\ > 0. 

Since (y - Ab, Av) = v'A'(y - Ab) and supp(v) = iV e c , 

\(y-Ab,Av)\ = \v' Ni A' N c(y-Ab)\ < M^A^y- Ab)^ 

Thus, 

\(y-Ab,Av)\ < max|(j/-A6,A w )|||t;||i (21) 

The third term of (T% can be written as 

\(Au,Av)\ < WA'AulUM, < SWA'AIUMU (22) 

And j|w||i = ||wt c ||i since supp(v) = N° C T c . Therefore, 

L(b + h)-L(b) > [ 7 -maJt|(y-i46 > A,)|-<y||i4 / A|| 00 ]||t;|| 1 

(23) 

Since we can select <5 > as small as possible, then we just 
need to have 

7- max\(y-Ab,A u )\ >0 (24) 

Invoke Lemma 1, we have An s (cn c — &i\r e ) = 
jMA A (A' A MA A )- 1 g A . Since y - Ab = (y - A Ne c Ne ) + 
An c {°n c — bN c ), therefore, 

\{y-Ab,A u )\ < \(y-A N 

+j\((A' A MA A y 1 A' A MA UJ ,g A )\ (25) 

Then we only need to have the condition 

7 - max [ 1 \{{A' A MA A )- l A' A MA u ,g A )\ + 

\(y-A Ne c Ne ,A u> )\]>0 (26) 

Since y — A^^cm^ is orthogonal to A w for each ui E N e , then 
max^TVe \(y - A Ne c Ne , AJ)\ = \\A'(y - A Ns c N JWoo. Also, 
we know that ma,yi UJ ^ N J((A' A MA A y 1 A' A MA UJ ,g A )\ < 
max^^ ||(A a A/Aa)~ iC A a A/^||i]. Thus, <|26]l holds if the 
following condition holds 

WA'iy-A^cvJWoo < T [l-max \\{A' A M A A )~ l A' A M A^] 

(27) 

i.e. b is the unique global minimizer if (|27| | holds. 
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